Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Database SQL load #3

Merged
merged 2 commits into from
Oct 29, 2023
Merged

Adds Database SQL load #3

merged 2 commits into from
Oct 29, 2023

Conversation

madflojo
Copy link
Member

@madflojo madflojo commented Oct 29, 2023

This code adds logic to load the database from the downloaded CSV file contents.

Summary by CodeRabbit

  • New Feature: Improved build process by adding a new build step to compile the load.wasm file using TinyGo.
  • New Feature: Enhanced the Airport Lookup Service by introducing a new service for data loading, which fetches airport data, parses it using a CSV parser, and updates the database with the parsed data. This also includes creating necessary database structures.
  • Improvement: Updated logging messages and levels in the data fetch function to provide clearer information and improve visibility.
  • Improvement: Added a new option to start services without building, allowing developers more flexibility in the development process.
  • Chores: Added a cleanup step in the build process to remove the functions/build directory and stop all services, ensuring a clean workspace.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 29, 2023

Walkthrough

The changes primarily focus on enhancing the build process, refining logging levels, and restructuring the data fetching and loading mechanism in the Airport Lookup Service. The modifications also include the addition of a new Go package and updates to the database initialization process.

Changes

File(s) Summary
Makefile A new build step has been added to compile load.wasm using TinyGo. Changes to the docker-compose target now start the mysql and airport-lookup-example services. New targets run-nobuild and clean have been added for starting services without building and for cleanup, respectively.
functions/src/data/fetch/main.go Log messages and levels have been updated for better clarity and information.
functions/src/data/load/main.go A new Go package has been added. It includes a Function struct with a Handler method for fetching, parsing, and updating airport data in the database.
functions/src/init/main.go The import statements have been updated, and a new SQL query has been added to create the "airports" table. The function call to fetch airport data has been replaced with a function call to load airport data. Error handling and logging statements have also been updated.

🐇💻

Code changes here, code changes there,

In the world of code, there's always something to repair.

With every push and every pull,

Our project grows, and our hearts are full.

So here's to the devs, with their coffee cups,

Coding away, until the sun comes up. 🌞🎉


Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai help to get help.
  • @coderabbitai resolve to resolve all the CodeRabbit review comments.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.json

@codecov
Copy link

codecov bot commented Oct 29, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (878ca02) 92.92% compared to head (d37ed15) 92.92%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main       #3   +/-   ##
=======================================
  Coverage   92.92%   92.92%           
=======================================
  Files           2        2           
  Lines          99       99           
=======================================
  Hits           92       92           
  Misses          5        5           
  Partials        2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 3

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between e4b3efd and 3d57de4.
Files ignored due to filter (8)
  • config/tarmac.json
  • docker-compose.yml
  • functions/src/data/load/go.mod
  • functions/src/data/load/go.sum
  • functions/src/init/go.mod
  • functions/src/init/go.sum
  • go.mod
  • go.sum
Files selected for processing (4)
  • Makefile (2 hunks)
  • functions/src/data/fetch/main.go (1 hunks)
  • functions/src/data/load/main.go (1 hunks)
  • functions/src/init/main.go (2 hunks)
Files skipped from review due to trivial changes (2)
  • Makefile
  • functions/src/data/fetch/main.go
Additional comments: 4
functions/src/init/main.go (2)
  • 1-6: The removal of unused imports is a good practice as it helps to keep the code clean and maintainable. Ensure that these removed packages are not used elsewhere in the code.

  • 12-45: The function Handler is responsible for initializing the Airport Lookup Service, creating the MySQL database structure if it doesn't exist, and loading the airport data. The error handling and logging are done appropriately. However, consider using parameterized queries or prepared statements to prevent SQL injection attacks, even though the current query does not seem to be vulnerable.

stmt, err := f.tarmac.SQL.Prepare(query)
if err != nil {
    f.tarmac.Logger.Error(fmt.Sprintf("Failed to prepare statement - %s", err))
    return []byte(""), fmt.Errorf("Failed to prepare statement: %s", err)
}
_, err = stmt.Exec()
if err != nil {
    f.tarmac.Logger.Error(fmt.Sprintf("Failed to create table - %s", err))
    return []byte(""), fmt.Errorf("Failed to create table: %s", err)
}

The function call to load airport data is done correctly. Ensure that the function "load" exists and is implemented correctly.

functions/src/data/load/main.go (2)
  • 1-8: Imports look good. No unused imports detected.

  • 14-37: The error handling and logging in the Handler method are well implemented. The function returns immediately when an error is encountered, and the error is logged with a descriptive message.

functions/src/data/load/main.go Outdated Show resolved Hide resolved
functions/src/data/load/main.go Outdated Show resolved Hide resolved
functions/src/data/load/main.go Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 3d57de4 and d37ed15.
Files selected for processing (1)
  • functions/src/data/load/main.go (1 hunks)
Additional comments: 4
functions/src/data/load/main.go (4)
  • 1-9: Imports look good. No unused imports detected.

  • 15-23: The error handling here is good. It logs the error and returns it.

  • 25-38: The error handling here is good. It logs the error and returns it.

  • 112-126: The error from the SDK initialization is not handled. As per the previous discussion, if the SDK fails to initialize, the logger might not be available. In such a scenario, you could consider returning an error from the main function, which would then be handled by the runtime. This approach might require changes in the way your program is run, depending on your runtime environment.

functions/src/data/load/main.go Show resolved Hide resolved
@madflojo madflojo merged commit 9008497 into tarmac-project:main Oct 29, 2023
4 checks passed
@madflojo madflojo deleted the parsecsv branch October 29, 2023 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant